An Objective Structured Clinical Exam on Breaking Bad News for Clerkship Students: In-Person Versus Remote Standardized Patient Approach

Introduction Telemedicine training for medical students is critical as that modality becomes integral to patient care. This formative standardized patient (SP) objective structured clinical exam (OSCE) lets students discuss miscarriage diagnosis and treatment virtually. Methods The SP OSCE was a mandatory session during the obstetrics and gynecology clerkship. Students received immediate feedback and optional individual reviews with clerkship directors. Students completed a nonmandatory survey at the end to describe their experience. SPIKES protocol student responses (i.e., proportion of correct responses) from in-person and remote SP versions were compared. Results Between July 2019 and March 2020, 79 students completed the in-person OSCE. Between July 2020 and June 2021, 149 students completed the remote SP encounter OSCE. Students who participated in the remote versus the in-person OSCE were more likely to admit their lack of knowledge when not equipped (p = .02), be seated during the encounter (p = .03), show listening body language (p = .13), assess the SP's perception (p = .19) and understanding (p = .20), and correct the SP's misunderstandings (p = .14). Of 84 students from eight rotations, including both in-person and remote formats, 99% believed learning objectives were clear, 91% felt preparation material was adequate, 95% thought the instructor summarized important points, 97% learned something in caring for gynecological patients, and 96% perceived the OSCE to be a worthwhile educational experience. Discussion The remote OSCE was well received by students. Breaking bad news virtually met assessment goals. Telemedicine training should be incorporated into medical school curricula.


Introduction
Early pregnancy loss, also called miscarriage or spontaneous abortion, occurs in 10% of clinically recognized pregnancies. Approximately 80% of cases occur within the first trimester. 1 This is a diagnosis seen in both fields of obstetrics and gynecology. Understandably, miscarriage can be distressing for the patient and requires more than a test results discussion. The consultation calls for communication that incorporates patience and empathy. In addition, providing counseling on management options including observation, medicine, or surgery is warranted. The Association of Professors of Gynecology and Obstetrics includes spontaneous abortion as an educational objective of the clerkship curriculum. 2 SPIKES (setting up interview, assessing patient perception, invitation, giving knowledge and information to patient, assessing patient emotions with empathic responses, and summary and strategy), a mnemonic that organizes the steps in delivering bad news, is commonly used in medical education. 3 SPIKES provides a six-step approach useful for a learner interviewing a patient, delivering results, and providing emotional support. During clerkship, a medical student may observe a health care provider breaking bad news in a clinical setting, but it is unlikely that a student would lead this discussion.
Standardized patient (SP) sessions have enhanced learner interaction and procedural skills in clinical fields, including obstetrics and gynecology. [4][5][6] Examples of breaking bad news communication curricula that utilize role-play and SPs are readily available. 7,8 The COVID-19 pandemic required health care systems to adapt patient care delivery models. Consequently, virtual care has become progressively widespread and valuable. 9 For example, telemedicine allows for an initial assessment of miscarriage through history taking alone to guide management decisions preceding the physical exam. The American Medical Association has recommended telemedicine training be incorporated into medical education since most students will experience telemedicine at some point in their career. 10 We developed a formative exercise as part of the obstetrics and gynecology clerkship curriculum in response to the COVID-19 pandemic. Given constraints on in-person activities, we converted our miscarriage objective structured clinical exam (OSCE) to a remote SP encounter. Participation in simulation through roleplay with SPs allowed students to practice delivering news of a miscarriage case and to discuss findings and management plans in a remote encounter. Aligned with domains III and IV of the AAMC telehealth competencies, 11 the exercise included an analysis of communication and history-taking skills. The SPIKES protocol was used to assess students' rapport with patients through eye contact and body language, as well as their approach in evaluating and managing a miscarriage case. 11 We compared results between the previous in-person and remote SP encounter groups to evaluate the impact on student performance of breaking bad news in a virtual setting.

Target Audience
Medical students at Weill Cornell Medicine participate in a 6week obstetrics and gynecology clerkship rotation. In July 2020, this was shortened to 4 weeks due to the COVID-19 pandemic. As part of the modified curriculum, the miscarriage OSCE was converted into a remote SP format. Students received an OSCE overview during orientation and were emailed instructions 3 days prior to the exercise. This information was available on their online learning website from day one of the rotation (Appendix A).
For OSCE preparation, students were provided with three articles on early pregnancy loss 1,12,13 and the SPIKES protocol, 3 accessible online throughout the rotation. Students could view the patient note (Appendix B) and post-follow-up exercise (Appendices C and D) online ahead of the session. During their preclinical training, students had participated in small-group sessions and informal role-play focusing on breaking bad news. Students received limited formal training on the approach to a telemedicine visit in the curriculum.

SP Preparation
SPs were referred by other SPs who had the required acting and teaching experience. The clerkship director and SP educator collaborated to create the OSCE script for SPs. Interested SPs received case details so they could make an informed decision. Thirteen SPs accepted the role. They spent 3 hours training with the SP educator during which they reviewed standardization and strategies to manage unanticipated learner questions and behavior. They practiced providing feedback and learned how to complete assessments. SPs were prepared through repeated practice of the case. 14 They received the SP training guide (Appendix E), which included case details, instruction, expected learner responses, and the patient note answer key. On the OSCE day, participating SPs met with the SP educator 30 minutes before students arrived. They received updates, asked questions, and discussed OSCE-related issues. Following the session, a debrief with the SP educator was held to give SPs time to derole.
Before conversion to the remote OSCE, SPs received a refresher session about the case. Zoom etiquette regarding punctuality, internet connection, lighting, privacy, background, microphone muting, and eating or drinking during the encounter was reviewed. SPs received webcams and were taught on correct positioning for effective eye contact. The day before, SPs received the Zoom link and login email, and all used computers at home as they had internet access.

Patient Encounter
Introduction: Seven to 10 clerkship students were on rotation every 4 weeks. The OSCE was a mandatory assignment scheduled on a set day. Students joined remotely from a location with privacy and an internet connection. Two OSCE sessions were run one after the other, each lasting 45 minutes. Students were divided between these groups. For example, if there were seven students, four were in the first group and three in the following. One SP was assigned to two students, one from each group, for the day. One SP role-played for both 45-minute sessions back to back, with a 5-minute break in between.
Students convened on Zoom in the main room. Here, the SP educator described OSCE steps. The scenario (Appendix A) portrayed a patient in the emergency room. After delayed menses, the patient had a positive home pregnancy test. Soon after, she experienced heavy bleeding and pelvic pain and was instructed by her gynecologist's office to go to the emergency room, where the SP interacted remotely with the learner, who was at a separate location.
The OSCE evaluator then met with the students. They reiterated OSCE steps and discussed the value of this opportunity to practice breaking bad news as well as the importance of developing this skill. The evaluator reviewed history-taking points, including use of open-ended questions, avoiding jargon, and how to be efficient. Evaluation included observation of the student-SP interaction, history-taking assessment, clinical decision-making, diagnosis, and management plan delivery. The evaluator had control on Zoom to move between breakout rooms during the session in order to observe all student-SP interactions simultaneously. A technician moved students between the main and breakout rooms. Students received an on-screen warning message when 2 minutes were left for each step. Steps: r The SP educator shared the first computer-screen door note with students in the main room (Appendix F).
r Initial encounter: Each student was moved to their individual breakout room to meet with their designated SP. They obtained a history and communicated relevant system exams and pelvic exam steps that would be performed in person. (15 minutes) r Students were moved back to the main room where they received the second computer-screen door note indicating pelvic exam findings, transvaginal ultrasound, and blood test results (Appendix G). Here, they completed the computer patient note (Appendix B). (10 minutes) r Follow-up: Students gathered their thoughts, returned to the breakout room with their SPs, delivered the diagnosis, and provided counseling and a management plan. (10 minutes) r Students returned to the main room to complete the postfollow-up exercise (Appendices C and D). (5 minutes) r Feedback: Students were moved back to their breakout rooms with their SPs for performance feedback. SPs used answers from the SPIKES protocol (Appendix H), history checklists (Appendix I), and the SP training guide (Appendix E) to provide students with an individualized and detailed assessment of their performance.
r The OSCE evaluator reconvened with students on Zoom to review answers for the patient note (Appendix E) and post-follow-up exercise (Appendices C and D), engage in discussion, and provide feedback on performance and comfort level during the encounter.
Learner Assessment Students received verbal feedback from SPs immediately after the session. An assessment of their history-taking skills and aptitude to formulate an accurate diagnosis and management plan was accomplished through observation by the OSCE evaluator, and feedback was provided at the debrief. Also, history-taking and communication skills were evaluated by SPs, who completed the history checklist (Appendix I) and SPIKES protocol questionnaire (Appendix H), a psychometrically valid survey. Each SPIKES question allowed for a yes or no response, and yes was the correct response for each question. The proportion of all students in a given rotation who answered each SPIKES question correctly was recorded for each question.
Additionally, students' answers on the post-follow-up exercise (Appendices C and D) and performance evaluation by SPs through the history checklist (Appendix I) were tabulated into numeric assessments and were available for optional individual review with clerkship directors. Patient note answers (Appendix E) were also available for discussion; however, they were not scored numerically. These scores and answers were not included in the data analysis.

OSCE Evaluation
Students provided informal feedback to the OSCE evaluator at the session debrief. They could opt to complete and submit a five-question survey rating the clarity of the learning objectives, preparatory material quantity, instructor's important points summary, exercise content utility and future patient care relevance, and educational experience (Appendix J). Students could also add free comments. The college formal clerkship evaluation was another platform where students provided feedback.

Data Analysis
The proportion of correct responses for each rotation, as defined above, was averaged across six rotations in the pre-COVID period (nine to 17 students per rotation) and 12 rotations in the post-COVID period (eight to 17 students per rotation). The average was weighted by rotation size and calculated for each SPIKES question separately. An exploratory statistical analysis was carried out for hypothesis-generating purposes. A twosample t test was used to compare the mean proportion correct for each SPIKES question (i.e., sat down, showed listening body language, etc.) between the pre-COVID time period (six weighted mean proportions) and post-COVID time period (12 weighted mean proportions). We considered p values less than .05 statistically significant. Any p value between .06 and <.20 was considered a nonsignificant trend since the sample size of the mean proportions correct was very small in the two time-period cohorts (N = 6 and N = 12) and, thus, analyses were considered likely to be underpowered. All analyses were performed in IBM SPSS Statistics version 27.

Results
Seventy-nine students completed the pre-COVID (in-person) OSCE during academic year 2019-2020 between July 2019 and March 2020. One hundred forty-nine students completed the post-COVID (remote) OSCE during academic year 2020-2021 between July 2020 and June 2021. There were fewer pre-COVID (in-person) groups (N = 6) because there were no in-person clerkship activities from April 2020 until July 2020 due to COVID. Also, the clerkship length was shortened from 6 to 4 weeks in the post-COVID (remote) groups, which increased rotation numbers to 12 in academic year 2020-2021.
The Table reflects mean proportion correct (weighted across six rotations pre-COVID and 12 rotations post-COVID) for each question on the SPIKES protocol assessment. There was no significant difference between the in-person and remote groups' performance while communicating with SPs about addressing their readiness to discuss test results, using understandable language during diagnosis discussion, allowing SPs to express emotions and responding with empathy, and discussing a management plan and asking SPs for questions. Students were more likely to admit a lack of knowledge when not equipped during the remote encounter compared to the in-person group (p = .02). A nonsignificant trend indicated students were more likely to assess SPs' understanding (p = .20) and perception (p = .19), show listening body language (p = .13), and correct SPs' misunderstandings (p = .14) also in the remote setting (Table).
Of eight rotations, there were 84 students who chose to complete the survey, including four pre-COVID (N = 52 students) and four post-COVID rotations (N = 32 students). Respectively, 98% and 100% agreed on clear learning objectives, 88% and 93% agreed on adequate preparation materials, 90% and 100% thought important points were summarized, 100% and 93% learned something useful in gynecological patient care, and 98% and 93% perceived the OSCE as a worthwhile educational experience.
Student survey free-response comments for the remote encounter included the following: r "This was awesome! Thank you." r "This was helpful and important to learn in an SP clinical setting because even in my short time on the rotation I saw a similar case with a resident." r "So happy to have the opportunity to become involved in a remote OSCE!" At the debrief with the OSCE evaluator, students described the remote encounter as emotional and felt uncomfortable breaking bad news because they had never done it before. The evaluator stated that the students' encounter was similar to their real telemedicine visits in that they had to learn how to connect, speak softly and empathetically, and allow for silence during conversation. The evaluator thought students overall displayed good body language and that, similar to doctors, some were more empathetic than others.
SPs stated that numerous students appeared uncomfortable while explaining the diagnosis during the in-person and remote OSCEs. Students who struggled frequently used noncommittal jargon in an attempt to minimize the intensity of the news. SPs collectively observed that while delivering news in either format, a majority of students adhered to the SPIKES steps. However, SPs noted that silence was easier on Zoom and that some students expressed empathy better than others. A few started crying during the remote encounter.

Discussion
The remote SP encounter OSCE accomplished curriculum goals. Students had a virtual simulated clinical experience and were evaluated on their knowledge, communication skills, and ability to exhibit empathy and compassion while delivering bad news. This platform provided data to compare with students' previous in-person OSCE performance.
Telemedicine is useful for the delivery of a sensitive diagnosis such as miscarriage and can prevent delay in management. Our activity gave students an opportunity to practice an OSCE case in a realistic learning environment, and overall, students provided positive feedback. Remote SP encounter survey results showed that students believed the learning objectives were clear (100%), the preparation material was adequate (93%), they learned something useful (93%), the experience was worthwhile (93%), and important points were well summarized by the evaluator (100%).
Using a remote format allowed students from multiple clinical sites to participate without the need to physically convene at one campus. In comparison with the in-person OSCE, the remote encounter group had similar scores for 50% of the domains generated through the SPIKES protocol checklist completed by the SPs. Students who participated in the remote encounter were more likely than those in the in-person group to admit their lack of knowledge when not equipped. Other checklist questions showed a nonsignificant trend. Students were more likely to show listening body language, correct the SPs' misunderstandings, and assess the SPs' perception and understanding during the remote encounter. The results for the item about being seated during the encounter were likely a function of the remote encounter format. The same SPs participated in the in-person and remote encounter OSCEs.
Results indicating that the remote encounter might be the more effective format for certain areas may merit additional investigation. It could be that for early trainees, a technology platform creates a safe distance, making it easier for a student to admit lack of knowledge and to correct misunderstandings. Also fascinating is that the SPs' perception of students showing better listening body language may be in part because Zoom forces visual focus on the person talking. An earlier study showed that patients expecting bad news wanted to learn results as quickly as possible and were more focused on message content than supportive aspects of communication, such as a hand on the shoulder. 15 Virtual visits reassure patients that providers are involved in their care, improving a patient's (or SP's) assessment of their provider. 16 Even in the business industry and remote working, visual is more effective compared to audio. Visual communication through videoconferencing builds camaraderie, productivity, and overall collaboration. 17 Telemedicine may lend itself to facilitating more advanced and uncomfortable conversations.
Students reported they felt uncomfortable breaking bad news. Data show that even during residency, a majority feel insufficient in conveying bad news to patients. 18 Good communication skills are important for an effective doctor-patient relationship and contribute to improved patient satisfaction and fewer lawsuits. 19 However, significant gaps in how physicians deliver bad news exist, and doctors acknowledge their lack of training. 18,20 In one study, only 47% of patients were pleased with how bad news was delivered to them, and this influenced their decisions regarding medical treatment and changing physicians in charge. 21 The need to continue teaching this skill at different medical training levels is recognized.
Limitations included our having a small sample size. At best, our data suggest the remote encounter is noninferior to the inperson OSCE as an effective platform for breaking bad news. However, not all AAMC telehealth competencies have been incorporated in this activity, so it is not an actual telemedicine simulation.
Other limitations included the following: There was back and forth between virtual rooms. Also, the OSCE evaluator moved between breakout rooms and occasionally would miss key portions of students' performance. While Zoom allows for recordings that can be viewed afterward, we suggest that sites replicating this project utilize three evaluators to eliminate concerns for incomplete assessments. We have modified our remote encounter so that students move fewer times between rooms (Appendix K).
We did not include survey questions specifically about the remote experience, which we intend to modify. Although SPs' feedback is reflected on the SPIKES checklist, in the future we can also document their verbal feedback to share with students and enhance their learning experience. Immediate post-OSCE feedback from SPs about students' empathy and rapport building has been well received by students in the literature. 22 This has been a useful exercise for students to practice delivering bad news in a virtual format. Our primary focus was to evaluate students' communication skills and expression of empathy and compassion. As virtual visits continue to become well accepted in medicine, addressing sensitive topics via telemedicine should be incorporated into the medical education curriculum at our institution. We will continue the OSCE as a remote encounter in the obstetrics and gynecology clerkship and incorporate AAMC telehealth competency domains into the structure to create a telemedicine simulation. Adaptations to the traditional SPIKES model, such as using more exaggerated body language and verbally confirming an alternative way to contact the SP in event of technologic failure, will also be included in our virtual training. 23